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Abstract 



We study the effects of dimension-six operators contributing to the gti vertex 
in top quark pair production at the Tevatron colhder. We derive both the 
hmits from Run 1 data and the potential bounds from future runs (Run 2 and 
3). Although the current constraints are not very strong, the future runs are 
quite effective in probing these operators. We investigate the possibility of 
disentangling different operators with the ti invariant mass distribution and 
the top quark polarization asymmetry. We also study the effects of a different 
set of operators contributing to single top production via the Wth coupling. 
We derive the current and potential future bounds on these anomalous opera- 
tors and find that the upgraded Tevatron can improve the existing constraints 
from for one of the operators. 
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Typeset using REVT^ 



I. INTRODUCTION 



The phenomenological success of the standard model (SM) has significantly limited the 
possibihty of new physics. However, some unanswered fundamental questions suggest that 
the SM will be augmented by new physics at higher energy scales. As the most massive 
fermion in the SM, the top quark is naturally regarded to be more sensitive to new physics 
than lighter fermions. Therefore, precision measurements of top quark properties offer one 
of the best possibilities to obtain information on new physics. 

The measurements of top quark properties in Run 1 at the Fermilab Tevatron have 
only small statistics. There is plenty of room for new physics to be discovered in similar 
measurements taken with higher luminosities. Such possibilities exist in the near future: 
Run 2 and Run 3 of the Tevatron collider should significantly improve the precision of the 
measurements of top quark properties and provide a better insight to new physics at 
higher energy scales. 

There are numerous speculations on the possible forms of new physics. The fact that no 
single clear signal for deviation from the SM has been observed in any experiment strongly 
suggests that the new physics effects not too far above the electroweak scale should preserve 
the SM structure and at most modify it delicately. This suggests that any new particles 
which may exist will be too heavy to be produced at current and quite possibly at near- 
future colliders, and thus the only observable effects of new physics at energies not too far 
above the SM energy scale could be in the form of anomalous interactions which will slightly 
affect the couplings of the SM particles. This reasoning leads to the effective Lagrangian 
approach in describing new physics effects. For the description of new physics in the top 
quark sector, there are two effective Lagrangian approaches, which are formulated in terms 
of non-linear and linear realizations of the electroweak symmetry, corresponding to 
the situations without and with a light Higgs boson, respectively. In this article, we choose 
the linear realization and parameterize the new physics effects in the top quark sector by 
dimension-six operators which are invariant under SU(3)c x SU(2)i x U(l)y. We will focus 
on the CP-conserving operators contributing to the anomalous gtt or Wtb coupling which 
can be directly probed at the upgraded Tevatron collider through top pair and single top 
production, respectively. 

The operators we consider describe the anomalous couplings of top quark with gauge 
and/or Higgs bosons and do not include top quark four-fermion [qqtt) contact terms, which 
have been investigated at Tevatron collider by Hill and Parke [§]. There have also been 
analyses of the phenomenology of an anomalous chromo-magnetic dipole moment gtt cou- 
pling at the Tevatron collider [^. The operators considered in this paper will also induce 
such an anomalous chromo-magnetic dipole moment coupling for gtt, but at the same time 
give rise to other kinds of anomalous couplings via operators that obey the SM symme- 
tries. Some of the operators contributing to the Wtb coupling also induce an anomalous 
Zbb coupling. Their effects on single top production cross section at the Tevatron were 
evaluated in Refs. |0 and |]TU|, with constraints derived from the earlier, less accurate data 
on Rh = T{Z bb)/r{Z — > hadrons). Our systematical analyses in this paper also include 
these operators, but with a more complete calculation by considering all possible back- 
grounds and by deriving the potential limits from the upgraded Tevatron. We find that the 
operators which contribute to the Zbb coupling are strongly constrained and can no longer 
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give rise to observable effects at the upgraded Tevatron due to the good agreement of Rb 
with the SM value. 

This paper is organized as follows. In Sec. |I|, we list the relevant dimension-six operators. 
In Sec. m we study the contribution of those operators to the gti coupling and their effect 
on top pair production. We derive the limits on the coupling strengths set by Run 1 and 
determine the limits which could be set by Run 2 and Run 3 if no new physics effects are 
observed. In Sec. |rV|, we present two methods to disentangle the effects between different 
operators contributing to the gtt coupling. One is studying the ti invariant mass distribution 
of the cross section, the other is measuring the asymmetry between left- and right-handed 
top events in top pair production. In Sec. |V| we evaluate the effects of those operators 
contributing to the Wtb coupling in single top production and derive the bounds which 
could be set by Run 2 and Run 3. Finally, Sec. |VI| is devoted to conclusions. 



II. LIST OF THE RELEVANT OPERATORS 



We assume here that the new physics in the quark sector only resides in the interaction 
of the third family to gauge bosons and/or Higgs boson. The effective Lagrangian including 
the new physics effects can be written as p[ 



A4 



(2.1) 



where Cq is the SM Lagrangian, A is the new physics scale, Oj are dimension-six operators 
which are SU(3)c x SU(2)i x U(l)y invariant, and Ci are coupling constants which represent 
the strengths of Oj. Recently, the effective operators involving the third-family quarks were 
reclassified in Refs. for the CP-conserving ones and in Ref. for CP-violating ones. 
There are 10 dimension-six CP-conserving operators which contribute to the gti or Wtb 
coupling 



OtG 
0,G 
OtG<S> 



G 



G 



G 



O, 



qW 



o 



(3) 



ObW'S' 
On 
Out 

Otw<!> 



inYT^D^ta + Wt^^T^tR 

{qLO^^^THR)^ + ^\tRa^^T\L] 
qLl^T'D'^qL+W^Ll^r^qj] W^, 



{q^D^tuW^ + {D^^)\D^tnqL) 



RO^^'r'qj:^ 



W, 



(2.2a) 
(2.2b) 
(2.2c) 
(2.2d) 
(2.2e) 
(2.2f) 
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where qi = {ti^hi) denotes the third family left-handed quark doublet, $ and $ are the 
Higgs field and its charge conjugate $ = ir2$*, W^i, and B^^, are the SU(2) and U(l) 
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gauge boson field tensors, respectively, in the appropriate matrix forms, denotes the 
appropriate covariant derivatives, and T"^ = with {A = 1, . . ., 8) denoting the 

Gell-Mann matrices. 

After electroweak symmetry breaking, the first three operators Otc, Ogc and Ojg<i> will 
induce anomalous gti couplings which are given by 
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(2.3) 



where v = {V2Gf)-^/'^ is the SM Higgs vacuum expectation value. Other operators will give 
rise to anomalous Wtb couplings which are given by 
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where g2 is the weak SU(2) gauge coupling. Note that the operators Ogiy, O^^, O^i, and 
ObVK'i' also induce Zhb couplings 
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where = cos6'm/, and thus will be subject to a constraint from i?^ measurements. 



III. TOP PAIR TOTAL CROSS SECTION 

The number of top pair events produced at the Tevatron will be different from the SM 
prediction if new physics exists in the top sector. Since the standard model cross section for 
top pair production is known, here we only need to evaluate the new physics contribution to 
the total cross section.^ We can predict the number of reconstructed top pair events in the 
various decay channels (dilepton, single lepton plus jets, etc.) at the upgraded Tevatron |1| 
by extrapolating from the Run 1 results. We will calculate the new physics contribution to 
the cross section and derive bounds on the coupling strengths of the anomalous operators. 
For illustration, we will only present the bounds at the 2a level. 



We assume that the new physics only affects the SM top quark couphngs and that there is no 
exotic decay mode for top quark. Therefore the branching fractions of various final states in ti 
production are about the same as in the SM. 
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A. Anomalous contribution to top pair production cross section 



The dominant mechanisms of top quark pair production at hadron coUiders are the QCD 
processes of quark-antiquark annihilation and gluon-gluon fusion. At the Tevatron colhder, 
the quark-antiquark annihilation process is dominant for = 175 GeV. In the SM, several 
groups pA|-p!3[] have calculated the cross section to the next-to-leading order by summing 
over soft gluons up to leading logarithms. More recently, soft-gluon resummation at the 
next-to-leading logarithmic level has also been performed 



We calculate the new physics contribution to the quark-antiquark annihilation process 
and neglect its effect in the gluon-gluon fusion process since its contribution to the cross 
section is small at the Tevatron. For the on-shell t and t, we obtain the effective gtt coupling 
arising from the dimension-six operators 



J- gtt 



9r77 1 

'-k^l5)FA + T^iPt-PtrFM 



2mt 



(3.1) 



where pt and pt are the momenta of the outgoing top quark and anti-quark, respectively, 
and k = Pt + Pt is the momentum of the vector boson. The form factors are derived from 
the contributions of the new physics operators 
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The pseudoscalar coupling proportional to A;^75 is related to the axial vector coupling by 
current conservation. It gives negligible contribution (proportional to the initial quark mass) 
to the process. 

The new physics contribution to the parton- level cross section for qq — > tt to order 1/A^ 
is found to be 



27s 



[Fvm3-P') + FM{s)P' 



(3.3) 



where s is the center-of-mass energy squared for the parton-level process and /3 = (1 — 
4mj/s)^/^. Notice that the axial vector coupling does not contribute to the total cross 
section. We will discuss in Sec. how to measure this coupling. 

The new physics contribution to the total hadronic cross section for top quark pair 
production is obtained by 



TO dr 



st), 



(3.4) 



where s is the pp center-of-mass energy squared, s the center-of-mass energy squared for the 
parton subprocess, and tq = imf/s. The quantity dLqq/dr is the parton luminosity defined 
by 



5 



^ = / ^[/n^i'/^)/f(^M'/^) + (9^ (3-5) 

where denotes the quark distribution function in a proton. 

In our numerical calculation, we use the CTEQ3L parton distribution functions |T5| with 
fi = For rrit = 175 GeV, we obtain the new physics contribution to order 1/A to the 
total hadronic cross section in unit of pb 

r [-OM{CtG + C,g) + O.SlCiG*] (A/TeV)-2 at v^i = 1.8 TeV, 
Aa^r = { (3-6) 
i [-O.SbiCtG + C,g) + 1.09^0*] (A/TeV)-2 at = 2 TeV. 



B. Current bounds from Run 1 

Current bounds for the coupling strength of the operators can be derived from the avail- 
able data on the cross section at Run 1 of the Tevatron. The production cross section mea- 



-1.5 

pb for rrit = 175 GeV, which combines dilepton, lepton + jets and all-hadronic channels. 



sured by the CDF collaboration with an integrated luminosity of 110 pb is [0 a = 7.6 



The DO collaboration gives [l^JT8[] cr = 5.9±1.7 pb for rrit = 172 GeV. We use the (unofficial) 



combined result iTBI of 



a'fP = 6.7± 1.3pb. (3.7) 
For the theoretical cross section in the SM, we adopt the most complete result currently 



available |jT^, which includes soft-gluon summation up to the next-to-leading logarithmic 
order: 

ar = 5Mtlil ph (3.8) 

for rrit = 175 GeV at ^/s =1.8 TeV. To make a comparison of the measured and standard 
model cross section, we have to take into account the present uncertainty of the top quark 
mass |T9| , ^ , p!8| which affects both the experimental and theoretical cross sections. Shifting 
rrit by ±5 GeV changes the SM cross section by about ±15%. The measured cross section 
also changes with rrit in the same direction but the dependence is weaker. The net effect is 
about 10% uncertainty in the difference of the two cross sections. 

Combining all these uncertainties, the possible new physics contribution to the cross 
section is found to be 

Aa^r = t^'"" - = 1-6 ± 1-4 pb, (3.9) 
which gives a 2a bound on the coupling strengths for A = 1 TeV 

- 7.2 < CtG + C,G - 1.33aG* < 2.0 . (3.10) 
If one of the dimension-six operator gives the dominant new physics contribution, we find 

-7.2 <CtG,C,G< 2.0 (3.11) 
-1.5 < CtG* < 5.4 . (3.12) 

We can see that the current bounds from Run 1 are not very strong. If one defines the new 
physics scale A such that the magnitude of the coupling strengths are 1, then Run 1 has 
excluded the existence of new physics below 400-800 GeV. 
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C. Expectations for Run 2 and 3 



To determine the size of the couphngs that can be probed in the upgraded Tevatron, we 
must estimate the number of top events produced. In our analysis, Run 2 and Run 3 are 
defined as an integrated luminosity of 2 and 30 fb~^ at ^/s = 2 TeV. We use the SM cross 
section for ti production of 7.0 pb at y/s = 2 TeV. At Run 2 (Run 3), the total number 
of the produced ti pairs is thus about 10"^ (2 x 10^). A detailed analysis of the detection 
efficiencies and signal purity for the three modes, the dilepton lepton plus >3 jets with 
one 6-tag (£3j/6), and lepton plus >4 jets with two 6-tags (£4j/26) can be found in Ref. 
They are shown in Table |. 

In extracting the new physics contribution, various systematic uncertainties have to be 
taken into account besides the experimental systematic error. The present uncertainty in 



the theoretical ti cross section in the standard model is at the 5% level [T^. Additional 
error coming from the present error on rrit will be much reduced with the expected more 
precise determination of nit (2.8 and 0.8 GeV are quoted for Run 2 and 3 [Q). We use the 
total systematic error of 5% and 1% for illustration. We also assume the same detection 
efficiencies as the SM events for the new physics contribution. 

Assuming no signal of new physics, the expected 2a bound on the coupling strength of 
the operators are obtained as shown in Table |T|; bounds are listed for systematic uncertain- 
ties of both 5% and 1%. A large improvement in the systematic error does not lead to a 
corresponding decrease in the bound at Run 2 because much of the error there still comes 
from the statistical uncertainty. At Run 3 the error is dominated by the systematic uncer- 
tainty and the bound would be substantially smaller if the systematic uncertainty could be 
reduced to 1%. For Cta = Cqc = CtG<s> = 1, these bounds correspond to the new physics 
scale A between 1 TeV and 2.5 TeV. Although Run 2 can improve the bounds from those at 
Run 1, one needs a better understanding of the theoretical uncertainty for further significant 
improvement of the bound at Run 3. 



IV. DISENTANGLING DIFFERENT OPERATORS 

In the preceding section we assumed the existence of one operator at a time and derived 
the bound for each operator from the event-number counting experiments at the future runs 
of the Tevatron. But the counting experiments cannot distinguish the effects of different 
operators. If the operators coexist, their effects have to be disentangled by analyzing ad- 
ditional measurable quantities. Here we present two methods to distinguish the effects of 
different operators contributing to the gti coupling; one is the ti invariant mass distribution 
of the cross section, and the other is the asymmetry between left- and right-handed top 
events in top pair production. 



A. tt invariant mass distribution 

The contribution of Otc and Ogc is energy dependent and thus the behavior of the cross 
section versus the invariant ti mass differs from that predicted by the SM. On the other 
hand, the contribution of OtG<s> is energy independent and gives the same M(ti) distribution 
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as the SM. This provides a method to distinguish the effects of Otc and/or Oqc from that 
of OtG^. 

The mass distribution is given by 



da _ d{a^^ + Aa 



dMtt dMtt 

= ^ (a^^ + Aa""'-) ^(r = M,|/s). (4.1) 

Figure 1 shows this distribution with and without the new physics contribution. It can be 
seen that the presence of OtC'p only alters the magnitude of the cross section, leaving the 
shape unchanged. The effect Otc is energy- dependent and is more prominent for larger Mu, 
giving some distortion in the invariant mass distribution. The result for Oqc is the same as 
that for OtG and is not shown. 

In order to quantify the shape of the distribution, we define the ratio of the high-mass 
vs. low-mass tt events as follows: 

^ N{Mu > Mq) ^ a{Mu > Mp) 

- N{Mu<Mo) a{Mu<M^y ^ ' ' 

where N{Mtt > Mq) is the number of events for Mu > Mq, etc. The value of Mq may be 
chosen to maximize sensitivity to the new physics. The ratio R is found to be a function of 
CtG + CqG- It thus provides independent information on new physics not contained in the 
total cross section. In particular, R will be the same as the SM prediction if only OtG<i> is 
present. 

Figure 2 shows the ratio R versus the coupling strength with and without the presence 
of OtG- We have used Mq = 600 GeV for illustration. We note that measuring this ratio 
will not only distinguish OtG (or Ogc) from OtG<;> but also determine the sign of the coupling 
constant Ctc (or C^g). 



B. Top polarization asymmetry 

In Sec. |Tl|, we have evaluated the new physics contribution to the total top pair events 
summing over the spins of t and i. Measurement of top polarization can give independent 
information for the new physics. In fact, the axial vector coupling Fa in Eq. (p.l|) contributes 
with the opposite sign to left- and right-handed top quark cross sections, producing a nonzero 
polarization asymmetry without changing the total cross section. 



Following Ref. we define the asymmetry as 



^-^F^ = ^^' (4.3) 
Nr + Nl aR + aL 

where Nr and Nl are the number of right- and left-handed top quarks, respectively. We 
do not require spin information on the i. The polarization may be measured^ through the 



^Anomalous tbW couplings may change the decay angular distribution. However, the three oper- 
ators considered in this and the previous sections do not produce such couplings. 
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angular distributions of the leptonic events t W^h — > f+z/^fo {£ = e.jj). In the 
SM this asymmetry is too small to be observed at the Tevatron, because QCD is invariant 
under parity and charge conjugation, and so the asymmetry arises only from the weak 
corrections [^. Hence the asymmetry is a good observable for probing new physics. 



The cross sections in (^]3|) can be written as ctr = am + (^rr and = ctlr + cll, 
with crAiA2 = <y{pp t\it\2 + Here Ai and A2 indicate the helicity states for t and i, 
respectively. These are obtained by convoluting the parton-level cross section a{qq — tt) 
with parton distribution functions as in Eq. ( |3.4| ). The parton-level cross section is expressed 
as 

a,,,,=al'f,^ + Aaill, (4.4) 

where (3"f '^^"1X2 physics contribution, respectively. At tree level, 

these are given by 

^LL = (^RR = -1^^ (4- 5a) 

27s s 

^LR = ^RL = o^7a ' (^-^b) 



and 



^^Tr = ^RR = ^ + I^FmJ aft, (4.6a) 

Aar^ = -{Fy- (3Fa) aft, (4.6b) 



9. 



s 



^^RL =-{Fv + f3FA) aft- (4.6c) 
9s 

Here we see that Aa^*^ differs from Ao"^'^ due to the existence of F4, which will cause the 
asymmetry A. 

To order 1/A^, we can neglect the new physics contribution to the denominator in 
Eq. (|4.3|). Again we use the CTEQ3L parton distribution functions with /i = 
For rrit = 175 GeV, we obtain the asymmetry 

A = -OmSiCtG - C,g) (A/TeV)-2. (4.7) 

A cut on the tt invariant mass > Mq enhances the asymmetry but at the same time 
decreases the number of ti events. 

The expected number of reconstructed ti events with top decay in the channel t — »• 
W^b —>■ t^V(b {£ = e, n) may be found in Table ||, which implies the la sensitivity for the 
asymmetry of ~ 3% and ~ 1% in magnitude at Run 2 and Run 3, respectively. Under the 
present constraints Eq. (|3.11| ), the operators Ctc or Cgo can produce an asymmetry A as 



large as 30% assuming no cancellation between these two operators. Such a large asymmetry 
should be clearly observable in future runs. If no asymmetry is observed, one can put bounds 
on the operators 
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for Run 2 (Run 3), which corresponds to ^ < 4/V (A^ is the total number of events). Al- 
though this bound is weaker than that obtained in Sec. |T|, it can be regarded as independent 
information because it does not depend on the coupling strength of 0^^$- 



Note that both of the observables in Sec. |IV A| and Sec. |IV B| distinguish the effects of 
OtG or OqG from that of Otc^- Furthermore, the two methods are complementary in that 
the ratio R in Eq. (|4.2|) depends on Ctc + Cqc while the asymmetry A in Eq. ( [4.3|) is sensitive 
to CtG — CqG- Therefore, separation of the effects of the three couplings is possible using 
the measurements discussed in this and the previous sections. 



V. SINGLE TOP QUARK PRODUCTION 



Now we examine the effects of the set of operators (|2.2d|) -( ^.2jP which contribute to 
single top production At the Tevatron, this reaction occurs mainly through the s-channel 
W* process q'q — ^ tb and the H^-gluon fusion process, which were studied extensively in the 
SM 
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and some of its extensions |2^. The s-channel W* process, despite its relatively 
low cross section, is quite powerful for probing new physics because (i) the systematic error 
in the theoretical calculation of its cross section is small (its initial state effects can be 
measured in the similar Drell-Yan process q'q —>■ iu), (ii) it can be isolated from other single 
top production processes at the upgraded Tevatron by requiring that both jets in the final 
state be tagged as b jets [^]. The balance between statistics and systematics for the W* 
process gives the result that its cross section can be measured to about the same precision 
as that for full single top cross section. So we choose the s-channel W* process to probe the 
operators.^ 

Unlike the case of top pair production in which the efficiency for top selection can be 
predicted by extrapolating from Run 1 experience, we will calculate the number of single 
top signal and backgrounds by Monte Carlo simulation. We will also take into account of 
the fact that the top quark is polarized in its production. 



A. Signal and backgrounds 



For both t and b being on-shell, the new physics contribution to the Wtb vertex can be 
written as 



p/i _ _ 92 

wfb - ^ 



1^{kilPl + KirPr) +pt{K2LPL + K2rPr) + P^{k3lPl + HsrPr) 



(5.1) 



where Pl,r = (1 T 75)/2 and the form factors from new physics are given by 



A2 



■ V2mt (3) k 

'^tW^ r <_/^g — U, 
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(5.2a) 



^The anomalous Wtb couplings may also be probed at the Tevatron in the top quark decay t — > Wb 



with spin analysis |23| 
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(5.2c) 
(5.2d) 
(5.2e) 
(5.2f) 



where k = pt + Pi- 

The interference of the SM matrix element (A^sm) with the new physics contribution 
(A^new) for the W* process u + d ^ t + b is given by (neglecting KM mixing) 
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for ht^ + 



for hf — — 



(5.4) 



for matrix elements with top quark helicity ht — ±. Here s is the parton cm. energy squared 
and 



i = (Pu - Pt)' = -2 - - cosr) 

u = {Pd-Ptf = -\{s - ml){l + cosr) 



(5.5a) 
(5.5b) 



where 9* is the cm. scattering angle. We have neglected the bottom quark mass. With this 
approximation, only one helicity of the quarks other than the top participate in the process 
and the contribution of Kir {i — 1, 2, 3), and hence that of the operators Oi,w^-i Ot^, and 
Oob, drops out. 

For the W* process, we look for events with t W^h —>■ i'^ub {i = e, /i) and thus the 
signature is an energetic charged lepton, missing Et, and double 6-quark jets. We assumed 
silicon vertex tagging of the &-quark jet with 50% efficiency and the probability of 0.4% for 
a light quark jet to be misidentified as a 6-jet. The potential SM backgrounds are: (Bl) 
the same W* process in the SM, (B2) the quark-gluon fusion process qg q'th where q' is 
misidentified as a 6-jet, (B3) processes involving a b-quark in the initial state, qb tq' and 
gb tW, (B4) tt W-W+bb. (B5) Wbb, and (B6) Wjj. 

Let us discuss the backgrounds in turn. Background process (B2) contains an extra light 
quark jet and can only mimic our signal if the quark misses detection by going into the 
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beam pipe. In our calculation of the W^-gluon fusion process as a background, we impose 
ri{q') > 3 and PrW) < 10 GeV for the light-quark jet. The qb tq' background^ is greatly 
reduced by requiring double 6-tagging. The process gb tW can only imitate our signal 
if the W decays into two jets, where one jet is missed by the detector and the other is 
misidentified as a 6 quark, which should be negligible for the misidentification rate assumed 
here. Background process (B4) can mimic our signal if both W^s decay leptonically and 
one charged lepton is not detected, which we assumed to occur if |r/(£)| > 3 and pr^i) < 10 
GeV. Since we required two 6-jets to be present in the final state, the potentially large 
background process (B5) from Wjj is reduced to an insignificant level. In order to reduce 
the backgrounds Wbb and Wjj, we required the reconstructed top quark mass M{bW) to 
lie within the mass range \M{bW) — m^l < 30 GeV. 

To simulate the detector acceptance, we made a series of cuts on the transverse momen- 
tum (pt), the pseudo-rapidity (rj), and the separation in the azimuthal angle-pseudo rapidity 



plane {AR = y(A0)2 + (Ar])^) between a jet and a lepton or between two jets. The cuts 
are chosen to be 

Pt, Pt, Pt > 20 GeV, (5.6a) 

\Vbl <2.5, (5.6b) 

ARjj, ARje > 0.5. (5.6c) 

To make the analyses more realistic, we simulate the detector effects by assuming a Gaussian 
smearing from the energy of the final state particles, given by: 

AEIE={ 30%//E © 1% for leptons, 

' \80%/\/E©5% forhadrons, ^ ' ' 

where © indicates that the energy dependent and independent terms are added in quadrature 
and E is in GeV. 

We have explicitly calculated backgrounds (Bl) and (B2), and for the others used the 
analysis of Ref. After all cuts, the number of background events at Run 2 (Run 3) are 
found to be 38 (570), 2 (26), 1 (14), 4 (54), 60 (900) and 1 (21), respectively, for the six 
background processes described above. Here we see that after the cuts the backgrounds from 
processes (B2), (B3), (B4) and (B6) are negligibly small. The number of total background 
events is 106 (1590) for Run 2 (Run 3). After the cuts, the number of signal events from 
new physics is found to be 



S 



(9aty$ + ^Ctl - llCgH' + 0.3Ci3t)(A/(TeV))^', at Run 2 

(5.8) 

(135GvK$ + 69CjfJ - l^'bCqw + 4.5CB^)(A/(TeV))"^ at Run 3. 



^There is a large uncertainty in the relative strengths of the qb and qg contributions to the 
background since the h quark parton distribution is not a measured quantity, but is rather an 
entirely theoretical construction. Since both the g6 and qg contributions after all cuts are a very 
small fraction of the total background, this uncertainty will not affect our results. 
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The effects of Odi are much smaller than those of the other three operators since it does not 
contribute to the form factor Kil, whose contribution is found to be much larger than those 
of the other form factors. 



B. Improvement of the bounds with Run 2 and 3 

From the results of the preceeding subsection, we obtain the bounds on the coupling 
strength of the operators from Run 2 (Run 3) if the new physics events are not observed at 
the 2a level 



I a 



qW\ 



< 2.1 (0.50), (5.9a) 



{A/TeVy 

< 4.6 (1.2), (5.9b) 

< 2.6 (0.61), (5.9c) 

< 77 (18), (5.9d) 



1^(3), 

(A/TeV)2 

I Ctw<s> I 
(A/TeV)2 

ICptl 
(A/TeV)2 



where we again assumed the simple situation that cancellation among different operators 
does not take place. 

As mentioned in Sec. H, the operators OgWi O'^^, O^b and Obw<s> also affect the Zbb 
coupling and will be subject to an Rb constraint. The SM values of Rh and the latest 



experimental data are 28 



/2^^ = 0.2158, = 0.2170(9). (5.10) 

With the new physics contribution described in Eq. (p.5|) , Rh is given by 



Rb — Rb 



SM 



4:SwCwvmz f cwmz ^(3)\ ^6 + «b d5a/n 



(5.11) 



where we have neglected the bottom quark mass and thus the contributions of Obw<s> and 
Oob, which are proportional to nib/mz, drop out. At the 2a level, we obtain the bounds for 
the coupling strength by assuming that no cancellation between Oqw and O^^ takes place. 

Since the dimension-six operators give a bad high energy behavior, there is an energy 
scale above which they cease to be a valid description of new physics. Any process below the 
new physics scale A should not violate the unitarity limit. For a given A, this requirement 
can be translated to an upper limit on the coupling strengths C. These unitarity limits have 
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been worked out by Gounaris et al. in detail. For Odi and Ofiy*, the strongest limits are 
obtained from two-body ti scattering process. For A = 1 TeV, they are given by 

\Ctw^\ < 13.5, (5.13a) 
\CDt\ < 10.4. (5.13b) 

The limit on Cot is independent of A. That on Ctw^ becomes somewhat weaker for larger 
A. 

Comparing Eqs. ( |5.9| ) with Eqs. ( |5.12| ) and ( p.l3[ ), we find: 

1. For the operators Oqw and O^^ , future runs at the Tevatron cannot improve the current 
bounds obtained from R^, which place much stronger constraints on these operators. 
Hence their effects will not be observable at the Tevatron, even for a luminosity of 100 
fb-i. 

2. The operator Otw<^^ currently subject only to weak bounds from unitarity, can be 
meaningfully probed at Run 2 and Run 3. 

3. The operator Odi cannot be probed at Run 2 and Run 3 much beyond the current 
bound from unitarity. For a higher integrated luminosity of 100 fb~^, we found that 
the bound is C/j^ < 9.8 which is only slightly stronger than its current bound. 

So we conclude that among the operators contributing to the Wth coupling, only one operator 
(OjvF*) can be meaningfully probed at the future runs of the Tevatron. 



VI. CONCLUSIONS 

We have studied in the effective Lagrangian approach the ability of future runs at the 
Tevatron in probing anomalous couplings of the top quark. We have listed and analyzed 
the possible dimension-six CP-conserving operators involving the anomalous gti or Wth 
couplings which could be generated by new physics at a higher scale. 

For the operators which give rise to an anomalous gti coupling, we evaluated their effects 
in top pair production and derive the bounds both from Run 1 and those expected from 
future runs. We found that the current constraints from Run 1 are not very strong and that 
future runs can either discover the effects of these operators or significantly improve the 
current bounds. We also proposed two methods to disentangle the effects between different 
operators contributing to the gtt coupling: one is studying the energy distribution of the 
cross section and the other is measuring the asymmetry between left- and right-handed top 
events in top pair production. 

For the operators which give rise to an anomalous Wth coupling, we calculated their 
contribution to single top production at the upgraded Tevatron and derived the bounds 
which could be obtained from Run 2 and Run 3. We found that future runs of the Tevatron 
cannot effectively probe those operators which are currently subject to a tight constraint from 
Rh- For the operators which are not subject to the Rh constraint and are only constrained 
by unitarity, future runs of the Tevatron can either discover the effects of or set stronger 
bounds on 0^^^$, while the best bound on Out is still obtained from the unitarity constraint 
for any new physics scale. 
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In Table |Tl| we summarize the strongest upper bounds on the operators under consid- 
eration that currently exist or can be obtained in the future at the Tevatron, for A = 1 
TeV. The current bounds are from our results using either i?^ or the Run 1 data at the la 
level, except for Ctw^ and Cdi where the best current bounds come from the requirement 

(3) 

of unitarity. We note that the current bounds on Cqw and C\,^ obtained from are better 
than those that can be obtained at Run 3 at the Tevatron. 
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TABLES 



TABLE I. Expected numbers of events and statistical errors for top pair production at Run 2 
and 3. The combined statistical error is derived from the U and ^3j/6 channels. 









Run 2 




Run 3 




Mode 


Efficiency 


Purity 


^events 


Statistical 


^events 


Statistical 




(%) 


S:B 


(with bkgd) 


error (%) 


(with bkgd) 


error (%) 


U 


1.2 


5:1 


200 


8.5 


3000 


2.2 




8.6 


3:1 


1600 


3.3 


24000 


0.9 




3.8 


12:1 


570 


4.5 


8600 


1.2 


total 








3.1 




0.8 



TABLE II. Expected 2a bounds at Run 2 and 3 for the operators contributing top pair pro- 
duction. If the three operators coexist, the limits for \CtG\ etc. should be reinterpreted to those 
for \CtG + CgG-1.28CtG^\. 







Run 2 






Run3 




Systematic error 


5% 




1% 


5% 




1% 


\CtG\, \CqG\ 


0.97 




0.53 


0.82 




0.21 


\CtG^\ 


0.75 




0.41 


0.64 




0.17 



TABLE III. Current and future strongest upper bounds on the magnitude of coupling strength 
of some operators at 2a level for A = 1 TeV. The current bounds are from unitarity requirement 

(3) 

for Ctw^ and Cot, from Rf, for Cgw and C$„ , and from Run 1 data for CtG, CqG and CtG^- 



Current bounds Bounds from Run 2 Bounds from Run 3 

IQcl, \CqG\ 1^0 08 

\CtG^\ 5.4 0.8 0.6 

\Cqw\ 0.2 2.1 0.5 

\C^^l\ 0.03 4.6 1.2 

\CtwA 13.5 2.6 0.6 

\CDt\ 10.4 77 18 
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FIGURES 
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FIG. 1. Top pair invariant mass distribution in the SM (solid), with contributions from Otc^ 
(dashed) and with Otc and/or Oqc (dotted and dash-dotted). The two curves for Otc/Oqc corre- 
spond to different couphng strengths. 
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FIG. 2. The ratio R with Mq = 600 GeV as a function of (Cjg + CgG)/A^- The SM value is 
indicated by the dashed line. 



20 



